Technical Field
Color image processing and display, and in particular, rendering of color images that are perceived to be of high aesthetic quality.
Description of Related Art
Current techniques for the rendering of high quality color images include techniques for the sharpening of images. Sharpening of an image is needed when the image has some level of blur. The blur may be caused by camera optics, and/or the image receptor in a digital camera, and/or the lack of steadiness of the camera when the image was acquired. At some level, image blur will be sufficiently high so as to cause a human observer of the image to perceive the image to be of low quality. Additionally, sharpening may be needed because blur in an image prevents a human observer or a computer algorithm from obtaining information from the image that would otherwise be perceivable or available if the image were sufficiently sharp.
Image and video processing to increase edge sharpness and reduce noise has been used to improve perceived quality since the inception of digital imaging due to the general implementation advantages of processing digital pixels in software or digital hardware over processing analog image and video data with complex analog electronics. Current image processing methods are directed to separate sharpening and noise reduction, with noise reduction most typically occurring before sharpening in order to reduce the visibility of sharpened image and video noise.
Current algorithms for image sharpening include the “unsharp mask” (UM), digital enhancement filters, and edge based sharpening. Although these algorithms function to sharpen an image, because they preserve local mean values of input still images or video image sequences, they produce unnatural edge transitions. By way of illustration, FIGS. 5A-5C depict a blurred edge in an unprocessed image, the blurred edge sharpened according to a prior art sharpening algorithm, and the blurred edge sharpened according to a method and algorithm of the present invention respectively. It can be seen that the edge of FIG. 5B processed with the prior art algorithm has a “dip” on the low value side of and edge and a “peak” on the high value side of the edge, also called undershoot and overshoot. These are seen as unnatural to viewers and often referred to as glows around edges because they do not represent the type of edge transition from better optics that viewers are familiar with visually. Such glows are perceived negatively by viewers as artifacts in the output images. In contrast, the edge of FIG. 5C has been processed with an algorithm and method of the present invention, as will be described subsequently herein. It can be seen that the edge is narrower, i.e., it has been sharpened, while also not having any overshoot or undershoot.
Current algorithms such as the unsharp mask also have the undesirable effect of enhancing image noise. Images having enhanced noise have poor aesthetics, i.e., they are perceived by observers to be “unpleasant to look at.” There are variations of the unsharp mask algorithm, such as weighted UM, but these operate such that the weights are dependent upon local pixel mean or edge intensity. Each of these variations of the UM algorithm has its own disadvantages and results in overshoot of edges or other artifacts.
Additionally, certain currently practiced algorithms and methods have been complex to implement. They may include spatial frequency decomposition of image or video data, with high frequency data treated as noise, and reduced and low frequency data treated as edges, Hence, these methods are not suitable for real time high resolution video image processing. Instead, their use is limited to still image processing.
Examples of noise reduction in currently practiced and algorithms include, but are not limited to, median filtering, statistical estimation, epilson non-linear gradient filters, and digital smoothing filers. These methods have been shown to be effective for reducing image and video noise but they also reduce edge sharpness, resulting in a need to have a combined method that sharpens edges and reduces noise that can be implemented in real time for high resolution video data.
In an attempt to address this need, color transformations have been used to calculate the image or video pixel intensity for the sharpening and noise reduction processing to avoid changing color pixel data. These methods used well known color transformations to convert input RGB image or video data to Cie L*a*b* and Cie Luv with L being intensity, or HSV with H being intensity, or processed input RGB data directly with transformation that isolated the G luminance data. Since changing L*, L, H or G also changes color hue and saturation, however, these methods cause color shifts along edges and in noise areas, both of which are perceived negatively by viewers as unnatural artifacts.
Furthermore, it is noted that most of the currently practiced algorithms are not suitable for software implementation as they require computation of mean or standard deviation around the “pixel of interest.” Current digital cameras have high resolution CCD sensors that have a very large number of pixels; even in cell phones, cameras having 8 megapixel CCDs are now common. With the pixel count of common digital images now being so high, a software implementation of current image sharpening algorithms is computationally prohibitive, especially using the processor in a cell phone, and with the desire by a user to perform image sharpening in a matter of seconds.
Thus there remains a need for an image sharpening method that can sharpen a color image without enhancing noise in the image, and that is not computationally intensive such that it can be executed on a processor of modest capability and achieve the sharpening in an acceptably short time, and in particular, at a short enough time to enable real time sharpening of images at video of movie sequence rates. Additionally, there is a need for an image sharpening algorithm and method that is able to suppress noise in an image. Additionally, there is a need for an image sharpening algorithm and method that does not cause color shifts along edges and in noise areas.